average performance
For SALE: State-Action Representation Learning for Deep Reinforcement Learning
We extensively study the design space of these embeddings and highlight important design considerations.
Appendix: BoostingOut-of-distributionDetection withTypicalFeatures
The mistaken rejection of an actually true null hypothesisH0 is the type I error.
LifelongPolicyGradientLearning ofFactoredPolicies forFasterTrainingWithoutForgetting
We provide a novel method for lifelong policy gradient learning that trains lifelong function approximators directly via policygradients, allowing the agent to benefit from accumulated knowledge throughout the entire training process.
A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm Qi Wang
Meta learning is a promising paradigm to enable skill transfer across tasks.
CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception
Carvalho, Miguel, Dias, Helder, Martins, Bruno
Vision-Language Models (VLMs) often struggle with tasks that require fine-grained image understanding, such as scene-text recognition or document analysis, due to perception limitations and visual fragmentation. To address these challenges, we introduce CropVLM as an external low-cost method for boosting performance, enabling VLMs to dynamically ''zoom in'' on relevant image regions, enhancing their ability to capture fine details. CropVLM is trained using reinforcement learning, without using human-labeled bounding boxes as a supervision signal, and without expensive synthetic evaluations. The model is trained once and can be paired with both open-source and proprietary VLMs to improve their performance. Our approach delivers significant improvements on tasks that require high-resolution image understanding, notably for benchmarks that are out-of-domain for the target VLM, without modifying or fine-tuning the VLM, thus avoiding catastrophic forgetting.
SAFE-SMART: Safety Analysis and Formal Evaluation using STL Metrics for Autonomous RoboTs
Sakano, Kristy, An, Jianyu, Manocha, Dinesh, Xu, Huan
We present a novel, regulator-driven approach for post hoc safety evaluation of learning-based, black-box autonomous mobile robots, ensuring ongoing compliance with evolving, human-defined safety rules. In our iterative workflow, human safety requirements are translated by regulators into Signal Temporal Logic (STL) specifications. Rollout traces from the black-box model are externally verified for compliance, yielding quantitative safety metrics, Total Robustness Value (TRV) and Largest Robustness Value (LRV), which measure average and worst-case specification adherence. These metrics inform targeted retraining and iterative improvement by model designers. We apply our method across two different applications: a virtual driving scenario and an autonomous mobile robot navigating a complex environment, and observe statistically significant improvements across both scenarios. In the virtual driving scenario, we see a 177% increase in traces adhering to the simulation speed limit, a 1138% increase in traces minimizing off-road driving, and a 16% increase in traces successfully reaching the goal within the time limit. In the autonomous navigation scenario, there is a 300% increase in traces avoiding sharp turns, a 200% increase in traces reaching the goal within the time limit, and a 49% increase in traces minimizing time spent near obstacles. Finally, we validate our approach on a TurtleBot3 robot in the real world, and demonstrate improved obstacle navigation with safety buffers.